feat(databricks): amend-and-build runs the dbt job now and waits for it - #861

…written, plan refuses a blocked residency (#868)

* fix(redact): mask a JWT by its JOSE header, never a fully qualified table name

fluid apply printed "databricks.load table=***REDACTED*** rows=340" for
northwind_towers.bronze_smart_meters.smart_meters and the real name for a
table whose last part had 11 characters. The prefix-agnostic _JWT_RE masked
any three dotted runs of 12 or more characters, and a catalog.schema.table
name (or project.dataset.table, DB.SCHEMA.TABLE) has that shape.

Such a run is now masked only when its first part is a JOSE header: base64url
of a JSON object, the check detect-secrets' JwtTokenDetector.is_formally_valid
makes (Yelp/detect-secrets plugins/jwt.py). The eyJ-anchored rule, which
gitleaks' jwt rule and trufflehog's JWT detector also anchor on, now covers a
JWE (a direct-encryption one has an empty encrypted key) and short or empty
payloads and signatures. One redact_jose_tokens() serves both redaction
layers: the Snowflake twin calls it, as it imports _URL_USERINFO_RE, so they
cannot drift. The rules start where no base64url character precedes them
instead of at \b, which matched after every '-' and made the scan quadratic
on a-a-a-... input (200k characters: about 30 s before, 0.07 s now).

keep_dotted_names goes: logs and printed errors apply the same rule now.

* fix(databricks): write Unity Catalog tag values as the contract states them

The governance labels became Unity Catalog tags with GCP's label slugs:
lowercased, spaces and punctuation as '-', cut at 63 characters
(business_value = 'a-single-unrenewed-ground-lease-can-strand-a-multi-tenant-
tower', fluid_jurisdiction = 'us'). A UC tag value takes up to 256 characters
with case, spaces and punctuation, and may not start or end with a space
(docs.databricks.com, "Apply tags to Unity Catalog securable objects").

governance_labels() gains verbatim_values: the Databricks module asks for the
contract's text, stripped, with the regulatory frameworks joined by ", ", and
refuses a value over 256 as databricks-tag-value-too-long (one _too_long
refusal now serves the product tags too). Keys keep GCP's label alphabet,
each '-' an '_'; GCP labels and AWS tags are unchanged.

An existing deployment updates in place: databricks_entity_tag_assignment's
tag_value carries no RequiresReplace at provider v1.140.0 and Update sends
UpdateMask "tag_value" (entity_tag_assignment/resource_entity_tag_assignment.go),
and the address and import id depend on the key alone. Pinned by a fake-
workspace e2e (slug values, then the real ones: +0 ~2 -0, nothing replaced)
and observed on Free Edition: +0 ~17 -0, the table's rows untouched, verify
green on all 27 tags.

* feat(plan)!: refuse a contract its own sovereignty policy blocks, by default

fluid validate and fluid plan --check-sovereignty refused a contract whose
binding region its strict sovereignty denies ("Region 'us-west-2' is
explicitly denied by sovereignty policy"), but plain fluid plan wrote a plan
for it, and --check-sovereignty itself wrote the plan before it exited 1.

plan now runs the built-in policy engine, the checker validate runs, before
it plans, and refuses (exit 1, sovereignty_violation, no plan.json) whenever
validate would: strict findings, or a region in deniedRegions, which blocks in
every mode. It builds no provider, so a plain plan stays off the cloud SDKs
(Command Center's plan endpoint relies on that) and prints nothing when
nothing blocks. --check-sovereignty keeps asking the provider's hook and
printing the verdict, now before the write. --no-check-sovereignty
(argparse.BooleanOptionalAction) skips the check and logs a WARNING with the
event sovereignty_check_skipped, the shape of apply's --no-verify-plan-binding;
it changes nothing in apply, which still refuses the placement on Databricks.
The design follows HCP Terraform's policy enforcement levels: a mandatory
policy stops the run, and an override is explicit and recorded.

BREAKING CHANGE: fluid plan exits 1 and writes no plan for a contract its
sovereignty policy blocks; pass --no-check-sovereignty to plan it anyway.

* fix(redact,plan): a linear two-segment JWT rule, and --check-sovereignty never weaker

From the branch's own review.

- _JWT_TWO_SEGMENT_RE still started at \b, so eyJ-eyJ-... stayed quadratic
  (200k characters: 9.75 s through redact_secret_text, 0.30 s now), and the
  linear test fed eyJ_ (no \b after '_'), so it could not see it. The rule now
  starts where _JWT_ANCHORED_RE does; _JWT_RE takes two runs as well as three,
  so a two-segment token glued after '-' is still masked by its header. The
  test feeds eyJ- too.
- _JWT_ANCHORED_RE took up to two more segments greedily, so "got <jwt>.Next"
  lost "Next". It now takes a JWS (three segments) or a JWE (five).
- --check-sovereignty asked the provider's hook first, so a hook that reported
  nothing passed a contract the new default refuses. It now runs the default
  gate first. --no-check-sovereignty also says so on the console.
- The changelog no longer says a failing plan stage leaves no plan file: one
  an earlier run left at --out stays.

* fix(plan,redact,databricks): the defect scan's findings

- fluid plan's sovereignty refusal printed its findings through Rich markup,
  which took each finding's "[expose_id]" for a tag and dropped it; findings
  print literally now (here and in --check-sovereignty's report). The refusal
  also carries the findings as the CLIError's context errors and message, the
  shape local_plan_validation_failed has: Command Center's plan endpoint
  surfaces an engine refusal that has them, and fell back to an ungated
  parse-only plan for one that had not.
- _JWT_RE takes up to five segments, empty ones included, so a JWE with
  direct encryption whose header is written with a space or a newline is
  masked whole again (origin/main masked its last three segments; the branch
  left it). Its header check still decides, so names stay readable.
- governance_labels' verbatim values spell a YAML true/false as written and
  make no tag for null: fluid generate iac emits without the schema gate,
  which allows only strings, and wrote Python's "True".

* chore(secrets): refresh the baseline's line numbers after the redactor's edits

* fix(redact): find a glued JOSE header in any segment of a dotted run

From the security judge's re-check. A token glued after a dotted name
(northwind_towers.x-<header>.<payload>) and a header written with spaces or
newlines glued to a word (token_eyAi..., x-ewog...) were masked on origin/main,
whose rule masked any three 12+ runs, but not on the branch, which looked for
a header only in the run's first segment and only at "eyJ".

_mask_if_jose now checks every segment of the run and masks from the first
that holds a header, keeping the name before it. A glued header starts at an
'e' after '-' or '_' or at an "eyJ" (base64url of '{' always starts with 'e'),
six tries at most per segment. _JWT_RE takes the whole dotted run, and
_is_jose_header requires the "alg" member RFC 7515 and RFC 7516 (4.1.1) make
mandatory, so a short JSON part of a name such as e30 ({}) is no header.
200,000-character adversarial runs stay under 0.5 s.

* fix(databricks): the job-run cleaner drops keep_dotted_names, now the default

#861's clean() asked redact_secret_text for keep_dotted_names=True, the
keyword this branch removes: every log and printed error now keeps a dotted
name and masks a JWT by its header, so the call passes nothing extra. Without
this, every Databricks job-run message raised TypeError (39 tests in
tests/build_runners/test_databricks_job_run.py after rebasing onto #861).

* style: one import style per module in the new tests; explicit concatenation in the catalog

From the code-quality review of #868. The two sovereignty_violation suggestions
split over adjacent literals now use the file's own parenthesised "+" form, and
the tests import fluid_build.observability.secret_redactor, the Snowflake logging
module and fluid_build.cli.plan one way each. No behaviour change; the baseline
moves line numbers only.

* test(redact): isolate the JOSE tests from secrets other tests registered

Test (Python 3.14) failed test_the_words_after_a_token_are_kept with
"got ***REDACTED******REDACTED***.Next step". The exact-match registry is
process-wide, and a masking test in the same xdist worker had registered the
base64url of bytes(range(32)), which is this test's signature, so layer 1
masked it first and the JOSE rule then masked "header.payload." on its own.
The token was masked whole either way; the exact-string expectation was not.

Both JOSE test classes now set the registry aside for each test and restore it
after (forget_known_secrets / register_secrets). Reproduced with that key
registered first: the old file fails 1, the new one passes 55.